The consistency of the BIC Markov order estimator

The Bayesian Information Criterion (BIC) estimates the order of a Markov chain (with nite alphabet A) from observation of a sample path x 1 ; x 2 ; : : :; x n , as that value k = ^ k that minimizes the sum of the negative logarithm of the k-th order maximum likelihood and the penalty term jAj k (jAj?1) 2 log n: We show that ^ k equals the correct order of the chain, eventually almost surely as n ! 1, thereby strengthening earlier consistency results that assumed an apriori bound on the order. A key tool is a strong ratio-typicality result for Markov sample paths. We also show that the Bayesian estimator or minimum description length estimator, of which the BIC estimator is an approximation, fails to be consistent for the uniformly distributed i.i.d. process.